Electronic device including electrochromic device and method of controlling the electronic device

ABSTRACT

Disclosed is an electronic device including an electrochromic device and a method of controlling the electronic device. The method includes: acquiring a first image frame using a camera of the electronic device, turning off an infrared (IR) lighting of the electronic device based on the IR lighting being on; acquiring a second image frame using the camera while the IR lighting is off; and controlling a transmittance of the electrochromic device based on a brightness of the second image frame based on the first image frame being acquired with the IR lighting on.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No.PCT/KR2022/011203 designating the United States, filed on Jul. 29, 2022,in the Korean Intellectual Property Receiving Office and claimingpriority to Korean Patent Application No. 10-2021-0125188 filed on Sep.17, 2021, in the Korean Intellectual Property Office, the disclosures ofwhich are incorporated by reference herein in their entireties.

BACKGROUND 1. Field

The disclosure relates to an electronic device including anelectrochromic device and a method of controlling the electronic device.

2. Description of Related Art

Electrochromism refers to a phenomenon in which color changes reversiblydepending on an applied voltage and may be used to control atransmittance of an electrochromic device.

The electrochromic device may be used as a lens of an electronic device,such as, for example, augmented reality (AR) glasses. The electronicdevice may sense an ambient brightness using an illuminance sensor andcontrol the transmittance of the electrochromic device based on theambient brightness.

An electronic device (e.g., augmented reality (AR) glasses) including anelectrochromic device may need to identify an ambient brightness tocontrol a transmittance of the electrochromic device so as to preventand/or reduce eyes of a user from experiencing glares.

The electronic device may include a camera configured to acquire animage to be used to track hands and a head of a user when providing anAR service and include an infrared (IR) lighting to acquire an imageusing the camera even when a surrounding environment is dark.

Although the electronic device identifies the ambient brightness usingan image acquired by the camera, it may not readily identify the ambientbrightness only using the camera because an amount of ambient light ofthe camera increases when the IR lighting is turned on in a darkenvironment. Thus, although the electronic device requires anilluminance sensor to accurately identify the ambient brightness, theinclusion of the illuminance sensor may increase the volume of theelectronic device and the cost used for the electronic device.

SUMMARY

According to an example embodiment, a method of controlling anelectronic device including an electrochromic device is provided, themethod including: acquiring a first image frame using a camera of theelectronic device; based on an infrared (IR) lighting of the electronicdevice being on, turning off the IR lighting; acquiring a second imageframe using the camera while the IR lighting is off; and based on thefirst image frame being acquired with the IR lighting on, controlling atransmittance of the electrochromic device based on a brightness of thesecond image frame.

According to an example embodiment, an electronic device including acamera, an infrared (IR) lighting, an electrochromic device, at leastone processor, and at least one memory configured to store thereinprocessor-implemented instructions is provided. The instructions, whenexecuted, may cause the processor to control the electronic device to:acquire a first image frame using the camera; turn off the IR lightingbased on the IR lighting being on; acquire a second image frame usingthe camera while the IR lighting is off; and based on the first imageframe being acquired with the IR lighting on, control a transmittance ofthe electrochromic device based on a brightness of the second imageframe.

According to an example embodiment, a method of controlling anelectronic device including an electrochromic device is provided, themethod including: acquiring an image using a camera of the electronicdevice; and based on the image being acquired while an infrared (IR)lighting of the electronic device being on, controlling a transmittanceof the electrochromic device based on a brightness of a first area inthe image not affected by the IR lighting.

According to an example embodiment, an electronic device including anelectrochromic device may acquire an image while an IR lighting is offfor an image frame that is not to be used for hand and head trackingamong multiple frames and use the acquired image to identify an ambientbrightness, thereby detecting the ambient brightness without beingaffected by the IR lighting even without an illuminance sensor andcontrolling the electrochromic device based on the detected ambientbrightness.

According to an example embodiment, an electronic device including anelectrochromic device may use an area in an image frame that is notaffected by an IR lighting to verify an ambient brightness, therebydetecting the ambient brightness without an illuminance sensor andcontrolling the electrochromic device based on the detected ambientbrightness.

Other features and aspects will be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing detailed description, taken in conjunction with theaccompanying drawings, in which:

FIG. 1 a perspective view illustrating an example structure of anelectronic device according to various embodiments;

FIGS. 2A, 2B, and 2C are diagrams illustrating an example camera and anexample infrared (IR) lighting of an electronic device according tovarious embodiments;

FIG. 3A is a diagram illustrating example image frames used in anelectronic device according to various embodiments;

FIG. 3B is a diagram illustrating an example of a temporal relationshipbetween an image frame used in an electronic device and an IR lightingaccording to various embodiments;

FIG. 4 is a flowchart illustrating an example method of controlling anelectronic device according to various embodiments;

FIGS. 5A and 5B are diagrams illustrating an example area affected by anIR lighting in an image acquired using a camera of an electronic deviceand an example area not affected by the IR lighting according to variousembodiments;

FIG. 6 is a flowchart illustrating an example method of controlling anelectronic device according to various embodiments; and

FIG. 7 is a block diagram illustrating an example configuration of anelectronic device according to various embodiments.

DETAILED DESCRIPTION

Hereinafter, various example embodiments will be described in greaterdetail with reference to the accompanying drawings. When describing theexample embodiments with reference to the accompanying drawings, likereference numerals refer to like elements and a repeated descriptionrelated thereto may not be provided.

FIG. 1 is a perspective view illustrating an example structure of anelectronic device according to various embodiments.

Referring to FIG. 1 , an electronic device 100 may be worn on a face ofa user to provide the user with an image associated with an augmentedreality (AR) service and/or a virtual reality (VR) service.

In an example embodiment, the electronic device 100 may include a firstdisplay 105, a second display 110, screen display portions 115 a and 115b, an optical input member 120, a first transparent member 125 a, asecond transparent member 125 b, lighting units 130 a and 130 b, a firstprinted circuit board (PCB) 135 a, a second PCB 135 b, a first hinge 140a, a second hinge 140 b, first cameras camera 145 a, 145 b, 145 c, and145 d, a plurality of microphones (e.g., a first microphone 150 a, asecond microphone 150 b, and a third microphone 150 c), a plurality ofspeakers (e.g., a first speaker 155 a and a second speaker 155 b), abattery 160, second cameras 175 a and 175 b, a third camera 165, andvisors 170 a and 170 b.

In an example embodiment, a display (e.g., the first display 105 and thesecond display 110) may include, for example, a liquid crystal display(LCD), a digital mirror device (DMD), or a liquid crystal on silicon(LCoS), an organic light-emitting diode (OLED), a micro light-emittingdiode (micro-LED), or the like. Although not illustrated in thedrawings, when the display is one of an LCD, a DMD, and an LCoS, theelectronic device 100 may include a light source configured to emitlight to a screen output area of the display. In an example embodiment,when the display is capable of generating light by itself, for example,when the display is either an OLED or a micro-LED, the electronic device100 may provide a virtual image of a relatively high quality to the usereven though a light source is not included. For example, when thedisplay is implemented as an OLED or a micro-LED, such a light sourcemay be unnecessary, and accordingly the electronic device 100 may belightened. The display capable of generating light by itself may bereferred to herein as a “self-luminous display,” and the followingdescription will be made on the assumption of the self-luminous display.

In an example embodiment, the display (e.g., the first display 205 andthe second display 210) may include at least one micro-LED. For example,the micro-LED may express red (R), green (G), and blue (B) by emittinglight by itself, and a single chip may implement a single pixel (e.g.,one of R, G, and B pixels) because the micro-LED is relatively small insize (e.g., 100 μm or less). Accordingly, the display may provide a highresolution without a backlight unit (BLU), when it is implemented by themicro-LED as described above.

However, examples are not limited thereto, and a single pixel mayinclude R, G, and B, and a single chip may be implemented by a pluralityof pixels including R, G, and B pixels.

In an example embodiment, the display (e.g., the first display 205 andthe second display 210) may include a display area including pixels fordisplaying a virtual image and light-receiving pixels (e.g., photosensorpixels) that are disposed between pixels and configured to receive lightreflected from eyes of a user, convert the received light intoelectrical energy, and output the electrical energy.

In an example embodiment, the electronic device 100 may detect a gazedirection (e.g., a movement of a pupil) of the user using thelight-receiving pixels. For example, the electronic device 100 maydetect and track a gaze direction of a right eye of the user and a gazedirection of a left eye of the user through one or more light-receivingpixels of the first display 105 and one or more light-receiving pixelsof the second display 110. The electronic device 100 may determine acentral position of a virtual image based on the gaze directions (e.g.,directions in which the pupils of the right eye and the left eye of theuser gaze) that are detected through the light-receiving pixels.

In an example embodiment, light emitted from the display (e.g., thefirst display 105 and the second display 110) may reach the screendisplay portion 115 a formed on the first transparent member 125 a thatfaces the right eye of the user and the screen display portion 115 bformed on the second transparent member 125 b that faces the left eye ofthe user, by passing through a lens (not shown) and a waveguide. Forexample, the light emitted from the display (e.g., the first display 105and the second display 110) may be reflected from a grating area formedin the optical input member 120 and the screen display portions 115 aand 115 b by passing through the waveguide, and may then be transmittedto the eyes of the user. The first transparent member 125 a and/or thesecond transparent member 125 b may be formed of, for example, a glassplate, a plastic plate, or a polymer, and may be transparently ortranslucently formed.

In an example embodiment, the lens (not shown) may be disposed in frontof the display (e.g., the first display 105 and the second display 110).The lens (not shown) may include a concave and/or convex lens. Forexample, the lens (not shown) may include a projection lens or acollimation lens.

In an example embodiment, the screen display portions 115 a and 115 b ora transparent member (e.g., the first transparent member 125 a and thesecond transparent member 125 b) may include a reflective lens, a lensincluding the waveguide.

The waveguide may be formed of glass, plastic, or a polymer, and mayhave a nanopattern formed on one surface of the inside or outsidethereof, for example, a grating structure of a polygonal or curvedshape. In an example embodiment, light incident on one end of thewaveguide may be propagated inside a display waveguide by thenanopattern to be provided to the user. For example, the waveguideformed as a freeform prism may provide the incident light to the userthrough a reflection mirror.

The waveguide may include at least one of a reflective element (e.g., areflection mirror) and at least one diffractive element (e.g., adiffractive optical element (DOE) or a holographic optical element(HOE)). The waveguide may guide light emitted from the display (e.g.,the first display 105 and the second display 110) to the eyes of theuser, using the at least one diffractive element or the reflectiveelement included in the waveguide.

In an example embodiment, the diffractive element may include theoptical input member 120 and/or an optical output member (not shown).For example, the optical input member 120 may refer to an input gratingarea, and the optical output member may refer to an output grating area.The input grating area may function as an input end to diffract (orreflect) light output from the display (e.g., the first display 105 andthe second display 110) (e.g., a micro-LED) to transmit the light to thetransparent member (e.g., the first transparent member 150 a and thesecond transparent member 150 b) of the screen display portions 115 aand 215 b. The output grating area may function as an outlet to diffract(or reflect), to the eyes of the user, light transmitted to thetransparent member (e.g., the first transparent member 150 a and thesecond transparent member 150 b) of the waveguide.

In an example embodiment, the reflective element may include an opticaltotal reflection element or a total reflection waveguide for totalinternal reflection (TIR). For example, total reflection or TIR, whichis one of schemes for inducing light, may form an angle of incidencesuch that light (e.g., a virtual image) input through the input gratingarea is completely or almost completely reflected from a portion (e.g.,a specific surface) of the waveguide, to completely or almost completelytransmit the light to the output grating area.

In an example embodiment, light emitted from the display (e.g., thefirst display 105 and the second display 110) may be guided by thewaveguide through the optical input member 120. The light traveling inthe waveguide may be guided toward the eyes of the user through theoptical output member. The screen display portions 115 a and 115 b maybe determined based on the light emitted toward the eyes of the user.

In an example embodiment, the first cameras 145 a, 145 b, 145 c, and 145d may include cameras used for three degrees of freedom (3DoF) and sixdegrees of freedom (6DoF) head tracking, hand detection and tracking,and gesture and/or spatial recognition. For example, the first cameras145 a, 145 b, 145 c, and 145 d may each include a global shutter (GS)camera to detect and track movements of a head or hand.

For example, the first cameras 145 a, 145 b, 145 c, and 145 d may use astereo camera for head tracking and spatial recognition, and may usecameras of the same specification and performance. For example, fordetection and tracking of a quick hand movement and a fine fingermovement, a GS camera exhibiting a favorable performance (e.g., imagedrag) may be used.

In an example embodiment, the first cameras 145 a, 145 b, 145 c, and 145d may use a rolling shutter (RS) camera. The first cameras 145 a, 145 b,145 c, and 145 d may perform spatial recognition for 6DoF and asimultaneous localization and mapping (SLAM) function through depthimaging. In addition, the first cameras 145 a, 145 b, 145 c, and 145 dmay perform a user gesture recognition function.

In an example embodiment, the second cameras 175 a and 175 b may be usedto detect and track the pupils. The second cameras 175 a and 175 b mayalso be referred to as an eye tracking (ET) camera. The second cameras175 a and 175 b may track a gaze direction of the user. Based on thegaze direction of the user, the electronic device 100 may dispose acenter of a virtual image projected onto the screen display portions 115a and 115 b at a position depending on a direction in which the pupilsof the user gaze.

The second cameras 175 a and 175 b for tracking the gaze direction mayuse a GS camera to detect the pupils and track a quick movement of thepupils. The second cameras 175 a and 175 b may be installed for the lefteye and the right eye of the user, respectively, and may use cameras ofthe same performance and specifications.

In an example embodiment, the third camera 165 may be referred to as a“high resolution (HR) camera” or a “photo video (PV) camera,” and mayinclude the HR camera. The third camera 165 may include a color camerahaving functions for acquiring a high-quality image, such as, forexample, an automatic focus (AF) function and an optical imagestabilizer (OIS). However, examples of the third camera 165 are notlimited thereto, and may include a GS camera or an RS camera.

In an example embodiment, at least one sensor (not shown)(e.g., a gyrosensor, an acceleration sensor, a geomagnetic sensor, and/or a gesturesensor), the first cameras 145 a, 145 b, 145 c, and 145 d may perform atleast one of head tracking for 6DoF, pose estimation and prediction,gesture and/or spatial recognition, and a SLAM function through depthimaging.

In an example embodiment, the first cameras 145 a, 145 b, 145 c, and 145d may be classified and used as a camera for head tracking and a camerafor hand tracking.

The lighting units 130 a and 130 b may be used differently according topositions to which the light units 130 a and 130 b are attached. Forexample, the lighting units 130 a and 130 b may be attached around ahinge (e.g., the first hinge 140 a and the second hinge 140 b)connecting a frame (e.g., a rim) and a temple, or be attached around afirst camera (e.g., 145 a, 145 b, 145 c, and 145 d) mounted adjacent toa bridge connecting the frame. For example, when a GS camera is used tocapture an image, the lighting units 130 a and 130 b may be used tosupplement a surrounding brightness. For example, the lighting units 130a and 130 b may be used in a dark environment or when it is not easy todetect an object to be captured due to a mixture or a reflection ofvarious light sources.

The lighting units 130 a and 130 b attached around the frame of theelectronic device 100 may be used as an auxiliary means for facilitatingeye-gaze detection when the pupils are captured using the second cameras175 a and 175 b. When the lighting units 130 a and 130 b are used as theauxiliary means for detecting the gaze direction, they may include an IRLED with an IR light wavelength.

In an example embodiment, a PCB (e.g., the first PCB 135 a and thesecond PCB 135 b) may include a processor including various processingcircuitry (not shown) configured to control components of the electronicdevice 100, a memory (not shown), and a communication module (notshown). The communication module may establish a direct (or wired)communication channel or wireless communication channel between theelectronic device 100 and an external electronic device, and supportcommunication through the established communication channel. The PCB maytransmit an electrical signal to the components included in theelectronic device 100.

The communication module (not shown) may include various communicationcircuitry including one or more communication processors that areoperable independently of the processor and that support direct (e.g.,wired) communication or wireless communication. According to an exampleembodiment, the communication module may include a wirelesscommunication module (e.g., a cellular communication module, ashort-range wireless communication module, or a global navigationsatellite system (GNSS) communication module) or a wired communicationmodule (e.g., a local area network (LAN) communication module or a powerline communication (PLC) module). A corresponding one of thesecommunication modules may communicate with an external electronic devicevia a short-range communication network, such as Bluetooth™,wireless-fidelity (Wi-Fi) direct, or infrared data association (IrDA))or a long-range communication network, such as a legacy cellularnetwork, a 5G network, a next-generation communication network, theInternet, or a computer network (e.g., a LAN or a wide area network(WAN)). These various types of communication modules may be implementedas a single component (e.g., a single chip), or may be implemented asmultiple components (e.g., multi chips) separate from each other.

The wireless communication module may support a 5G network after a 4Gnetwork, and a next-generation communication technology, e.g., a newradio (NR) access technology. The NR access technology may supportenhanced mobile broadband (eMBB), massive machine type communications(mMTC), or ultra-reliable and low-latency communications (URLLC). Thewireless communication module 192 may support a high-frequency band(e.g., a mmWave band) to achieve, e.g., a high data transmission rate.The wireless communication module 192 may support various technologiesfor securing performance on a high-frequency band, such as, e.g.,beamforming, massive multiple-input and multiple-output (MIMO), fulldimensional MIMO (FD-MIMO), an array antenna, analog beamforming, or alarge-scale antenna.

The electronic device 100 may further include an antenna moduleincluding at least one antenna (not shown). The antenna module maytransmit or receive a signal or power to or from the outside (e.g., anexternal electronic device) of the electronic device 100. According toan example embodiment, the antenna module may include an antennaincluding a radiating element including a conductive material or aconductive pattern formed in or on a substrate (e.g., the first PCB 135a and the second PCB 135 b). According to an example embodiment, theantenna module may include a plurality of antennas (e.g., arrayantennas).

In an example embodiment, a plurality of microphones (e.g., the firstmicrophone 150 a, the second microphone 150 b, and the third microphone150 c) may process an external sound signal into electrical audio data.The audio data may be used in various ways according to a function (orapplication) being performed (or executed) in the electronic device 100.

In an example embodiment, a plurality of speakers (e.g., the firstspeaker 155 a and the second speaker 155 b) may output audio datareceived from the communication module or stored in the memory.

In an example embodiment, the battery 160 may be provided as one or morebatteries, and may supply power to the components included in theelectronic device 100.

In an example embodiment, the visors 170 a and 170 b may adjust atransmitted amount of external light incident on the eyes of the userbased on a transmittance. The visors 170 a and 170 b may be disposed ona front or rear side of the screen display portions 115 a and 115 b. Thefront side of the screen display portions 115 a and 115 b may indicate adirection opposite to a user's side of the user wearing the electronicdevice 100, and the rear side of the screen display portions 115 a and115 b may indicate a direction of the user's side of the user wearingthe electronic device 100. The visors 170 a and 170 b may protect thescreen display portions 115 a and 115 b and adjust the transmittedamount of the external light.

For example, the visors 170 a and 170 b may each include anelectrochromic device that changes in color according to applied powerand adjusts the transmittance. Electrochromism refers to a phenomenon inwhich color changes in response to an occurrence of anoxidation-reduction reaction by applied power. The visors 170 a and 170b may adjust the transmittance of the external light using the colorchange of the electrochromic device.

For example, the visors 170 a and 170 b may each include a controlmodule and the electrochromic device. The control module may control theelectrochromic device to adjust the transmittance of the electrochromicdevice.

FIGS. 2A, 2B, and 2C are diagrams illustrating an example camera andexample IR lighting of an electronic device according to variousembodiments.

The electronic device 100 may be worn around a face of a user, providingimages associated with AR and/or VR services to the user. In an exampleembodiment, the electronic device 100 may include a plurality of camerasto provide the AR and/or VR services to the user. The cameras may sensewavelengths of visible and IR areas to acquire image frames, and theelectronic device 100 may perform head and hand tracking and recognize aspace using the acquired image frames.

For example, FIG. 2A illustrates the user wearing the electronic deviceand areas 205 and 210 illuminated by the cameras of the electronicdevice 100. For example, the electronic device 100 may include a frontcamera configured to acquire an image frame of the front area 205 and adownward-facing camera that faces the area 210 under the front area 205.

In an example embodiment, the electronic device 100 may include anelectrochromic device (e.g., the electrochromic device of FIG. 1 ) andcontrol a transmittance of the electrochromic device based on abrightness of an image frame acquired through a camera of the electronicdevice 100. For example, the electronic device 100 may control thetransmittance of the electrochromic device to be lower as the brightnessof the acquired image frame is higher so as to prevent and/or reduceeyes of the user from experiencing glares that may occur by lighttherearound (or ambient light hereinafter).

The electronic device 100 may acquire an image frame using the camera,and may not readily perform head tracking, hand tracking and spatialrecognition (e.g., scene understanding) using an image frame acquiredwhen a brightness around the camera (or ambient brightness hereinafter)is low, or a surrounding environment is dark). In an example embodiment,the electronic device 100 may include an IR lighting (e.g., the lightingunits 130 a and 130 b of FIG. 1 ) to acquire an image frame with abrightness required for head tracking, hand tracking, and spatialrecognition, and may secure a greater amount of light by turning on theIR lighting when the ambient brightness is low or the surroundingenvironment is dark. For example, FIGS. 2B and 2C are side and frontviews of the areas 205 and 210 illuminated by the camera of theelectronic device 100 and areas 215 and 220 illuminated by the IRlighting.

In an example embodiment, the camera may detect wavelengths of visibleand IR areas to acquire an image frame, and a brightness of an imageframe acquired while the IR lighting is on may be high (or bright) eventhough humans actually recognize a surrounding environment as beingdark. When controlling the transmittance of the electrochromic deviceusing an image frame acquired with the IR lighting on, the electronicdevice 100 may lower the transmittance of the electrochromic devicebecause a brightness of the image frame is high even though an actualenvironment around the user is dark and may thus restrict a view or afield of vision of the user. Thus, the electronic device 100 may requirea separate illuminance sensor to identify an ambient brightness that isrecognized by the user even when the IR lighting is on.

In an example embodiment, the electronic device 100 may acquire an imageframe by turning off the IR lighting for an image frame that is not tobe used for head tracking and hand tracking among image frames acquiredthrough the camera and verify (or identify) an ambient brightness usingthe acquired image frame, thereby detecting the ambient brightnesswithout being affected by the IR lighting even in the absence of anilluminance sensor.

Hereinafter, image frames used in the electronic device 100 and atemporal relationship between the image frames and an IR lighting willbe described in greater detail with reference to FIGS. 3A and 3B.

FIG. 3A is a diagram illustrating example image frames used in anelectronic device according to various embodiments, and FIG. 3B is adiagram illustrating an example temporal relationship between an imageframe used in an electronic device and an IR lighting according tovarious embodiments.

In an example embodiment, the electronic device 100 may include aplurality of front cameras and a plurality of downward-facing cameras torecognize a wider range. For example, as illustrated, the electronicdevice 100 may include a first front camera (e.g., the first camera 145a of FIG. 1 ), a second front camera (e.g., the first camera 145 b ofFIG. 1 ), a first downward-facing camera (e.g., the first camera 145 cof FIG. 1 ), and a second downward-facing camera (e.g., the first camera145 d of FIG. 1 ). The first front camera and the second front cameramay be respectively disposed on a left side and a right side of theelectronic device 100. The first downward-facing camera and the seconddownward-facing camera may be respectively disposed on the left side andthe right side of the electronic device 100.

The electronic device 100 may perform head tracking, hand tracking, andspatial recognition (e.g., scene understanding) using image framesacquired using the cameras (e.g., the first front camera, the secondfront camera, the first downward-facing camera, and the seconddownward-facing camera). The first front camera, the second frontcamera, the first downward-facing camera, and the second downward-facingcamera may sense wavelengths of visible and IR areas and acquire theimage frames.

The electronic device 100 may separately include a camera for headtracking, a camera for hand tracking, and a camera for spatialrecognition to acquire image frames to be used for head tracking, handtracking, and spatial recognition, and may be configured to operatethese cameras separately. In such a case, however, the volume of theelectronic device 100 may increase.

In an example embodiment, to perform head tracking, hand tracking, andspatial recognition, the electronic device 100 may only use the firstfront camera, the second front camera, the first downward-facing camera,and the second downward-facing camera, without separately including thecameras respectively for head tracking, hand tracking, and spatialrecognition, and may operate with a high frame rate to acquire imageframes and use the image frames for different purposes.

To perform head tracking, hand tracking, and spatial recognition,respectively, different frame rates may be required. For example, headtracking may require a frame rate of 24 frames per second (fps), handtracking may require a frame rate of 48 fps, and spatial recognition mayrequire a frame rate of 5 fps. As illustrated in FIG. 3A, the electronicdevice 100 may operate with 96 fps to perform all head tracking, handtracking, and spatial recognition, using only the first front camera,the second front camera, the first downward-facing camera, and thesecond downward-facing camera. However, a frame rate of image framesacquired from the first front camera, the second front camera, the firstdownward-facing camera, and the second downward-facing camera is notlimited to the foregoing example, and the frame rate may be determinedin various ways as needed.

The first front camera, the second front camera, the firstdownward-facing camera, and the second downward-facing camera may besynchronized by the same frame rate as illustrated in FIG. 3A to operateto acquire image frames. FIG. 3A illustrates example image framesacquired when the first front camera, the second front camera, the firstdownward-facing camera, and the second downward-facing camera operatewith 96 fps.

The image frames may be used for different purposes. The image framesillustrated in FIG. 3A may be used for head tracking, hand tracking, andspatial recognition. For example, as illustrated in FIG. 3A, among 96image frames acquired for a time interval 305 of 1 second, 48 imageframes may be used for hand tracking. Five image frames acquired by thefront cameras for a time interval 310 of the time interval 305 of 1second may be used for spatial recognition and head tracking. Inaddition, five image frames acquired by the downward-facing cameras forthe time interval 310 of the time interval 305 of 1 second may be usedfor head tracking. Further, 19 image frames acquired for a time interval315 of the time interval 305 of 1 second may be used for head tracking.

When the image frames are acquired by operating the first front camera,the second front camera, the first downward-facing camera, and thesecond downward-facing camera with a high frame rate as illustrated inFIG. 3A, there may be image frames that are not used for head tracking,hand tracking, and spatial recognition.

FIG. 3B illustrates some image frames 320 among the image frames of FIG.3A, and a state 325 of an IR lighting when each image frame is acquired.

Referring to FIG. 3B, the electronic device 100 may acquire first imageframes 327, 328, and 329 for a time interval 330. A first image framemay be an image frame that is to be used for at least one of handtracking, head tracking, or spatial recognition. An ambient brightnessbefore the time interval 330 may be determined to be low, or asurrounding environment therebefore may be determined to be dark, andthus the IR lighting may be turned on when the first image frames 327,328, and 329 are acquired for the time interval 330.

The electronic device 100 may acquire a second image frame 335 for atime interval 340. The second image frame 335 may be an image frame thatis not to be used for hand tracking, head tracking, and spatialrecognition.

When acquiring a second image frame (e.g., second image frames 335 and350) that is not to be used for hand tracking, head tracking, andspatial recognition, the electronic device 100 may acquire the secondimage frame (e.g., the second image frames 335 and 350) by turning offthe IR lighting regardless of whether the IR lighting is on before. Forthe time interval 340, the electronic device 100 may acquire the secondimage frame 335 by turning off the IR lighting that is on for a previoustime interval which is the time interval 330. The electronic device 100may control a transmittance of an electrochromic device of theelectronic device 100 based on a brightness of the second image frame335 acquired for the time interval 340. For example, as the brightnessof the second image frame 335 is higher, the electronic device 100 maycontrol the transmittance of the electrochromic device to be lower.

When at least one of the first image frames 327, 328, and 329 isacquired while the IR lighting is on, the electronic device 100 maydetermine a brightness difference between the at least one of the firstimage frames 327, 328, and 329 acquired while the IR lighting is on andthe second image frame 335 by comparing a brightness of the at least oneof the first image frames 327, 328, and 329 and a brightness of thesecond image frame 335.

The electronic device 100 may turn on or off the IR lighting based onthe brightness difference. For example, the electronic device 100 mayturn on the IR lighting when the brightness difference is greater thanor equal to a first threshold value, and turn off the IR lighting whenthe brightness difference is less than the first threshold value. Asmall brightness difference may indicate that a surrounding environmentis sufficiently bright, and thus the IR lighting may not be required toacquire an image to be used for hand tracking, head tracking, andspatial recognition.

In the example of FIG. 3B, the electronic device 100 may not turn on theIR lighting because the brightness difference between the at least oneof the first image frames 327, 328, and 329 and the second image frame335 is less than the first threshold value. When the electronic device100 does not turn on the IR lighting in the time interval 340, theelectronic device 100 may acquire first image frames 341, 342, and 343while the IR lighting is off in a time interval 345.

In a case where the IR lighting is off when acquiring a first imageframe (e.g., at least one of the first image frames 327, 328, 329, 341,342, and 343), the electronic device 100 may control the transmittanceof the electrochromic device based on a brightness of the acquired firstimage frame. For example, as the brightness of the first image frame ishigher, the electronic device 100 may control the transmittance of theelectrochromic device to be lower.

In a case where the IR lighting is off when acquiring a first imageframe (e.g., at least one of the first image frames 327, 328, 329, 341,342, and 343), the electronic device 100 may not turn on or off the IRlighting based on a brightness of the first image frame. For example,when the brightness of the first image frame is less than or equal to asecond threshold value, the brightness of the first image frame may notbe high enough to perform hand tracking, head tracking, and spatialrecognition, and thus the electronic device 100 may turn on the IRlighting to secure a greater amount of light.

The electronic device 100 may acquire a second image frame 350 for atime interval 355. In a case where the IR lighting is on when acquiringthe second image frame 350, the electronic device 100 may turn off theIR lighting and acquire the second image frame 350. When the first imageframes 341, 342, and 343 are acquired while the IR lighting is off, theelectronic device 100 may not use the second image frame 350 to controlthe electrochromic device and the IR lighting.

In an example embodiment, the electronic device 100 may control thetransmittance of the electrochromic device and control the IR lightingbased on a brightness of the second image frame 350 in the same way asperformed with the first images 327, 328, 329, 341, 342, and 343. Forexample, when the first image frames 341, 342, and 343 are acquiredwhile the IR lighting is off, the electronic device 100 may control thetransmittance of the electrochromic device to be lower as the brightnessof the second image frame 350 is higher. For another example, when thefirst image frames 341, 342, and 343 are acquired while the IR lightingis off, the electronic device 100 may turn on the IR lighting inresponse to the brightness of the second image frame 350 being less thanor equal to the second threshold value, so as to secure a greater amountof light.

In the example of FIG. 3B, the IR lighting may be off when a first imageframe 360 is acquired. A brightness of the first image frame 360 may beless than the second threshold value, and the electronic device 100 mayturn on the IR lighting based on the brightness of the first image frame360. The electronic device 100 may acquire a first image frame 365 whilethe IR lighting is on. Since the first image frame 365 is acquired whilethe IR lighting is on, the electronic device 100 may turn off the IRlighting and acquire a second image frame 375, and control theelectrochromic device based on a brightness of the second image frame375. The electronic device 100 may compare a brightness of at least oneof the first image frames 365 and 370 and a brightness of the secondimage frame 375 and determine a brightness difference therebetween. Theelectronic device 100 may control the IR lighting based on thebrightness difference.

In an example embodiment, the electronic device 100 may acquire a secondimage frame that is not to be used for hand tracking, head tracking, andspatial recognition while the IR lighting is off, and control thetransmittance of the electrochromic device and the IR lighting using theacquired second image frame, thereby reducing the volume of theelectronic device 100, without a separate illuminance sensor or aseparate camera for hand tracking, head tracking, and spatialrecognition.

Hereinafter, a method of controlling the electronic device 100 will bedescribed in greater detail with reference to FIG. 4 .

FIG. 4 is a flowchart illustrating an example method of controlling anelectronic device according to various embodiments.

Referring to FIG. 4 , in operation 405, the electronic device 100 mayacquire a first image frame using a camera (e.g., at least one of thefirst front camera, the second front camera, the first downward-facingcamera, or the second downward-facing camera of FIG. 2 ) In operation410, the electronic device 100 may verify whether an IR lighting is on.

In an example embodiment, when the first image frame (e.g., the firstimage frames 327, 328, and 329 of FIG. 3A) is acquired with the IRlighting on, the electronic device 100 may control an electrochromicdevice based on a brightness of a second image frame (e.g., the secondimage frame 340 of FIG. 3A). When a first image frame (e.g., the firstimage frames 341, 342, and 343 of FIG. 3A) is acquired with the IRlighting off, the electronic device 100 may control the electrochromicdevice based on a brightness of the first image frame (e.g., the firstimage frames 341, 342, and 343 of FIG. 3A).

In operation 415, when the first image frame is acquired with the IRlighting on, the electronic device 100 may turn off the IR lighting toacquire the second image frame. In operation 420, the electronic device100 may acquire the second image frame. The second image frame may be animage frame that is acquired with the IR lighting off and is notaffected by the IR lighting.

In operation 425, the electronic device 100 may control a transmittanceof the electrochromic device based on the brightness of the second imageframe. For example, as the brightness of the second image frame ishigher, the electronic device 100 may control the transmittance of theelectrochromic device to be lower.

In operation 430, the electronic device 100 may determine a brightnessdifference between the first image frame and the second image frame bycomparing the brightness of the first image frame and the brightness ofthe second image frame. In operation 435, the electronic device 100 maydetermine whether the brightness difference is greater than or equal toa first threshold value.

That the brightness difference is greater than or equal to the firstthreshold value may indicate that an amount of light increases byturning on the IR lighting when a surrounding environment is dark. Thus,in operation 440, the electronic device 100 may turn on the IR lightingto secure a greater amount of light for a subsequent image frame.

In an example embodiment, the electronic device 100 may turn on the IRlighting based on the brightness of the second image frame, withoutcomparing the brightness of the first image frame and the brightness ofthe second image frame. For example, when the brightness of the secondimage frame is less than or equal to a second threshold value, theelectronic device 100 may turn on the IR lighting.

In operation 445, when the brightness difference is less than the firstthreshold value, it may indicate that there is no significant differencein brightness of the images when the IR lighting is on because thesurrounding environment is sufficiently bright, and thus the electronicdevice 100 may maintain the IR lighting to be off.

In an example embodiment, the electronic device 100 may control thetransmittance of the electrochromic device based on the brightnessdifference determined in operation 430 without performing operation 425.For example, as the brightness difference determined in operation 430 issmaller, the electronic device 100 may control the transmittance of theelectrochromic device to be lower.

In operation 450, when the IR lighting is verified to be off inoperation 410, the electronic device 100 may control the transmittanceof the electrochromic device based on the brightness of the first imageframe. For example, as the brightness of the first image frame ishigher, the electronic device 100 may control the transmittance of theelectrochromic device to be lower.

In operation 455, the electronic device 100 may determine whether thebrightness of the first image frame is less than or equal to the secondthreshold value. In operation 460, when the brightness of the firstimage frame is less than or equal to the second threshold value, theelectronic device 100 may turn on the IR lighting to secure a greateramount of light.

When the brightness of the first image frame exceeds the secondthreshold value, there may be no need to secure a greater amount oflight, and thus the electronic device 100 may maintain the IR lightingto be off in operation 445.

In an example embodiment, when the first image frame is acquired withthe IR lighting off in operation 405, the electronic device 100 mayacquire the second image frame with the IR lighting off in the same wayas the foregoing case where the electronic device 100 acquires the firstimage frame with the IR lighting on, and then control the transmittanceof the electrochromic device based on a brightness of the second imageframe.

In an example embodiment, the electronic device 100 may iterativelyperform operations 405, 410, 415, 420, 425, 430, 435, 440, 445, 450,455, and 460 described above with reference to FIG. 4 . By performingoperations 405, 410, 415, 420, 425, 430, 435, 440, 445, 450, 455, and460 described above with reference to FIG. 4 , the electronic device 100may control the electrochromic device and the IR lighting based on anambient brightness without an illuminance sensor, and perform handtracking, head tracking, and spatial recognition.

Hereinafter, a method of controlling a transmittance of anelectrochromic device based on an area in an image frame will bedescribed in greater detail with reference to FIGS. 5A and 5B, and 6 .

FIGS. 5A and 5B are diagrams illustrating an example area affected by anIR lighting in an image acquired using a camera of an electronic deviceand an example area not affected by the IR lighting according to variousembodiments.

Referring to FIG. 5A, an image frame 505 may be acquired by a firstfront camera (e.g., the first camera 145 a of FIG. 1 ) and an imageframe 510 may be acquired by a second front camera (e.g., the firstcamera 145 b of FIG. 1 ). In an example embodiment, the first frontcamera and the second front camera may be respectively disposed on leftand right sides of the electronic device 100 as illustrated in FIG. 2C.

Referring to FIG. 5B, an image frame 535 may be acquired by a firstdownward-facing camera (e.g., the first camera 145 c of FIG. 1 ) and animage frame 540 may be acquired by a second downward-facing camera(e.g., the first camera 145 d of FIG. 1 ). The first downward-facingcamera and the second downward-facing camera may be respectivelydisposed on the left and right sides of the electronic device 100 asillustrated in FIG. 2C.

An IR lighting may be disposed in a middle portion of the electronicdevice 100 as illustrated in FIG. 2C. In an example embodiment, the IRlighting may be disposed on each of the left and right sides of theelectronic device 100.

In an example embodiment, an image frame acquired by the first frontcamera, the second front camera, the first downward-facing camera, andthe second downward-facing camera may include first areas 515, 530, 550,and 560 that are not affected by the IR lighting and second areas 520,525, 545, and 555 that are affected by the IR lighting, according to adisposition of the first front camera, the second front camera, thefirst downward-facing camera, and the second downward-facing camera. Forexample, referring to FIG. 2B, an area 225 may be included in an imageframe acquired by the first downward-facing camera and the seconddownward-facing camera, and may be out of a direction in which the IRlighting is illuminated and may thus not be affected by the IR lighting,and an area corresponding to the area 225 in the image frame may berelatively dark even when the IR lighting is on.

A first area (e.g., 515, 530, 550, and 560) and a second area (e.g.,520, 525, 545, and 555) may be adjacent to each other as illustrated inFIGS. 5A and 5B, but examples of which are not limited thereto, and thefirst area and the second area may be determined to have various formsin an image frame. For example, the first area may be a portion of thefirst areas 515, 530, 550, and 560 illustrated in FIGS. 5A and 5B, andthe second area may be a portion of the second areas 520, 525, 545, and555 illustrated in FIGS. 5A and 5B.

In an example embodiment, the electronic device 100 may control atransmittance of an electrochromic device based on a brightness of afirst area that is not affected by an IR lighting in an image frameacquired through a camera, and may thus not include a separateilluminance sensor.

Hereinafter, a method of controlling the electronic device 100 accordingto an example embodiment will be described in greater detail withreference to FIG. 6 .

FIG. 6 is a flowchart illustrating an example method of controlling anelectronic device according to various embodiments.

Referring to FIG. 6 , the electronic device 100 may acquire an imageframe using a camera (e.g., at least one of the first front camera, thesecond front camera, the first downward-facing camera, or the seconddownward-facing camera of FIG. 2 ) in operation 605.

In operation 610, the electronic device 100 may verify whether an IRlighting is on. In operation 615, when an image frame is acquired withthe IR lighting on, the electronic device 100 may control atransmittance of an electrochromic device based on a first area in theimage frame. The first area may be an area that is not affected by theIR lighting in the image frame, and the electronic device 100 maycontrol the transmittance of the electrochromic device based on thefirst area to control the transmittance of the electrochromic device tobe suitable for a field of vision of a user even when the IR lighting ison. For example, as a brightness of the first area is higher, theelectronic device 100 may control the transmittance of theelectrochromic device to be lower.

In operation 620, the electronic device 100 may determine a brightnessdifference by comparing the brightness of the first area in the imageframe and a brightness of a second area in the image frame. The secondarea may be an area affected by the IR lighting in the image frame. Whenan ambient brightness is high, the brightness of the image frame may notchange significantly even by turning on the IR lighting, and thus adifference between the brightness of the first area and the brightnessof the second area may not be great.

In operation 625, the electronic device 100 may determine whether thebrightness difference between the first area and the second area is lessthan or equal to a first threshold value. In operation 630, when thebrightness difference is less than or equal to the first thresholdvalue, the electronic device 100 may determine that the ambientbrightness is sufficiently high (or a surrounding environment issufficiently bright), and turn off the IR lighting. By turning off theIR lighting in operation 630, the electronic device 100 may acquire asubsequent image frame while the IR lighting is off.

In operation 635, when the brightness difference exceeds the firstthreshold value, the electronic device 100 may determine that theambient brightness is not sufficiently high (or the surroundingenvironment is not sufficiently bright), and maintain the IR lighting tobe on. By maintaining the IR lighting to be on, the electronic device100 may acquire a subsequent image frame while the IR lighting is on.

In an example embodiment, the electronic device 100 may control thetransmittance of the electrochromic device based on the brightnessdifference determined in operation 620 without performing operation 615.For example, as the brightness difference determined in operation 620 issmaller, the electronic device 100 may control the transmittance of theelectrochromic device to be lower.

In operation 640, when the IR lighting is verified to be off inoperation 610, the electronic device 100 may control the transmittanceof the electrochromic device based on a brightness of an entire area ofthe image frame. For example, as the brightness of the image frame ishigher, the electronic device 100 may control the transmittance of theelectrochromic device to be lower.

In operation 645, the electronic device 100 may determine whether thebrightness of the image frame is less than or equal to a secondthreshold value. For example, the electronic device 100 may determinewhether the brightness of the entire area of the image frame is lessthan or equal to the second threshold value.

In operation 650, when the brightness of the image frame is determinedto be less than or equal to the second threshold value in operation 645,the electronic device 100 may turn on the IR lighting to secure agreater amount of light. By turning on the IR lighting in operation 650,the electronic device 100 may acquire a subsequent image frame while theIR lighting is on.

In operation 635, when the brightness of the image frame is determinedto exceed the second threshold value in operation 645, the electronicdevice 100 may maintain the IR lighting to be off. By maintaining the IRlighting to be off by the electronic device 100, the electronic device100 may acquire a subsequent image frame while the IR lighting is off.

In an example embodiment, when the image frame is acquired with the IRlighting off, the electronic device 100 may control the transmittance ofthe electrochromic device based on the brightness of the first area inthe same way as in the foregoing case where the image frame is acquiredwith the IR lighting on.

In an example embodiment, the electronic device 100 may iterativelyperform operations 605, 610, 615, 620, 625, 630, 635, 640, 645, and 650described above with reference to FIG. 6 . The electronic device 100 mayperform operations 605, 610, 615, 620, 625, 630, 635, 640, 645, and 650described above with reference to FIG. 6 to control the electrochromicdevice and the IR lighting according to an ambient brightness, withoutan illuminance sensor, and perform hand tracking, head tracking, andspatial recognition.

Hereinafter, a configuration of the electronic device 100 will bedescribed in greater detail with reference to FIG. 7 .

FIG. 7 is a block diagram illustrating an example configuration of anelectronic device according to various embodiments.

Referring to FIG. 7 , according to an example embodiment, an electronicdevice 200 (e.g., the electronic device 100 of FIG. 1 ) may include aprocessor (e.g., including processing circuitry) 715 (e.g., theprocessor (not shown) of FIG. 1 ), a memory 720 (e.g., the memory (notshown) of FIG. 1 ), at least one camera 705 (e.g., the first cameras 145a, 145 b, 145 c, and 145 d), and at least one IR lighting 710 (e.g., thelighting units 130 a and 130 b of FIG. 1 ).

In an example embodiment, the processor 715 may include variousprocessing circuitry and acquire a first image frame through the camera705 to perform at least one of hand tracking, head tracking, or spatialrecognition.

The processor 715 may verify whether the IR lighting 710 is on. Forexample, when the first image frame (e.g., the first image frames 327,328, and 329 of FIG. 3A) is acquired with the IR lighting 710 on, theprocessor 715 may turn off the IR lighting 710 and acquire a secondimage frame (e.g., the second image frame 340 of FIG. 3A).

The processor 715 may control a transmittance of an electrochromicdevice based on a brightness of the second image frame to control atransmittance of the electrochromic device based on an image acquiredwithout an influence of the IR lighting 710. For example, as thebrightness of the second image frame is higher, the processor 715 maycontrol the transmittance of the electrochromic device to be lower.

To determine whether to turn on the IR lighting 710, the processor 715may determine a brightness difference by comparing the brightness of thefirst image frame and the brightness of the second image frame. Theprocessor 715 may turn on the IR lighting 710 when the brightnessdifference is greater than or equal to a first threshold value, andmaintain the IR lighting 710 to be off when the brightness difference isless than the first threshold value.

In an example embodiment, the processor 715 may control thetransmittance of the electrochromic device based on the brightnessdifference between the first image frame and the second image framewithout controlling the transmittance of the electrochromic device basedon the brightness of the second image frame. For example, as thebrightness difference between the first image frame and the second imageframe is smaller, the processor 715 may control the transmittance of theelectrochromic device to be lower.

When a first image frame (e.g., the first image frames 341, 342, and 343of FIG. 3A) is acquired with the IR lighting 710 off, the processor 715may control the transmittance of the electrochromic device based on thebrightness of the first image frame. For example, as the brightness ofthe first image frame is higher, the processor 715 may control thetransmittance of the electrochromic device to be lower.

The processor 715 may determine whether the brightness of the firstimage frame is less than or equal to a second threshold value. When thebrightness of the first image frame is less than or equal to the secondthreshold value, the processor 715 may turn on the IR lighting 710 tosecure a greater amount of light. When the brightness of the first imageframe exceeds the second threshold value, the processor 715 may maintainthe IR lighting 710 to be off.

When the first image frame is acquired with the IR lighting 710 off, theprocessor 715 may acquire the second image frame while the IR lighting710 is off and control the transmittance of the electrochromic devicebased on the brightness of the second image frame, in the same way as inthe case where the first image frame is acquired while the IR lighting710 is on.

In an example embodiment, the processor 715 may control theelectrochromic device and the IR lighting 710 using a first area in theimage frame that is not affected by the IR lighting 710 and a secondarea that is affected by the IR lighting 710.

The processor 715 may acquire an image frame through the camera 705 toperform at least one of hand tracking, head tracking, or spatialrecognition.

The processor 715 may determine whether the IR lighting 710 is on. Whenan image frame is acquired with the IR lighting 710 on, the processor715 may control the transmittance of the electrochromic device based ona first area in the image frame. For example, as a brightness of thefirst area is higher, the processor 715 may control the transmittance ofthe electrochromic device to be lower.

The processor 715 may determine a brightness difference by comparing thebrightness of the first area and a brightness of a second area in theimage frame to control the IR lighting 710. When the brightnessdifference is less than or equal to a first threshold value, theprocessor 715 may determine that an ambient brightness is sufficientlyhigh (or a surrounding environment is sufficiently bright), and turn offthe IR lighting 710. When the brightness difference exceeds the firstthreshold value, the processor 715 may determine that the ambientbrightness is still low (or the surrounding environment is still dark),and maintain the IR lighting 710 to be on to secure a greater amount oflight.

In an example embodiment, the processor 715 may control thetransmittance of the electrochromic device based on the brightnessdifference between the first area and the second area, not based on thefirst area. For example, as the brightness difference between the firstarea and the second area is smaller, the processor 715 may control thetransmittance of the electrochromic device to be lower.

In an example embodiment, when the image frame is acquired with the IRlighting 710 off, the processor 715 may control the transmittance of theelectrochromic device based on a brightness of the image frame. Forexample, the processor 715 may control the transmittance of theelectrochromic device based on a brightness of an entire area of theimage frame. In this example, as the brightness of the entire area ofthe image frame is higher, the processor 715 may control thetransmittance of the electrochromic device to be lower.

The processor 715 may determine whether the brightness of the imageframe is less than or equal to a second threshold value to control theIR lighting 710. When the brightness of the image frame is less than orequal to the second threshold value, the processor 715 may turn on theIR lighting 710 to secure a greater amount of light. When the brightnessof the image frame exceeds the second threshold value, the processor 715may maintain the IR lighting 710 to be off.

When the image frame is acquired with the IR lighting 710 off, theprocessor 715 may control the transmittance of the electrochromic devicebased on the brightness of the first area, in the same way as in thecase where the image frame is acquired with the IR lighting 710 on.

According to an example embodiment, a method of controlling theelectronic device including an electrochromic device may include:acquiring a first image frame using the camera of the electronic device,turning off an infrared (IR) lighting based on the IR lighting of theelectronic device being on, acquiring a second image frame using thecamera while the IR lighting is off, and controlling a transmittance ofthe electrochromic device based on a brightness of the second imageframe based on the first image frame being acquired while the IRlighting is on.

When the first image frame is acquired with the IR lighting on, themethod may further include: determining a brightness difference bycomparing a brightness of the first image frame and a brightness of thesecond image frame, and turning on the IR lighting based on thebrightness difference being greater than or equal to a first thresholdvalue.

When the brightness of the second image frame is less than or equal to asecond threshold value, the method may further include: turning on theIR lighting.

The controlling of the transmittance of the electrochromic device basedon the brightness of the second image frame may include: controlling thetransmittance of the electrochromic device to be lower as the brightnessof the second image frame is higher.

The controlling of the transmittance of the electrochromic device basedon the brightness of the second image frame may include controlling thetransmittance of the electrochromic device to be lower as the brightnessdifference is greater.

When the first image frame is acquired with the IR lighting off, themethod may further include turning on the IR lighting in response to thebrightness of the first image frame being greater than or equal to thesecond threshold value.

When the first image frame is acquired with the IR lighting off, themethod may further include controlling the transmittance of theelectrochromic device based on the brightness of the first image frame.

When the first image frame is acquired with the IR lighting off, themethod may further include controlling the transmittance of theelectrochromic device based on the brightness of the second image frame.

According to an example embodiment, an electronic device (e.g., 200) mayinclude a camera (e.g., 705), an infrared (IR) lighting (e.g., 710), anelectrochromic device, and at least one processor (e.g., 715), and atleast one memory (e.g., 720) storing therein instructions to be executedby the processor 715. The instructions, when executed may cause theprocessor to control the electronic device to: acquire a first imageframe using the camera; turn off the IR lighting based on the IRlighting being on; acquire a second image frame using the camera whilethe IR lighting is off; and, based on the first image frame beingacquired with the IR lighting on, control a transmittance of theelectrochromic device based on a brightness of the second image frame.

The instructions, when executed may further cause the processor to:based on the first image frame being acquired with the IR lighting on,determine a brightness difference by comparing a brightness of the firstimage frame and the brightness of the second image frame; and, turn onthe IR lighting based on the brightness difference being greater than orequal to a first threshold value.

The instructions, when executed, may further cause the processor tocontrol the electronic device to: turn on the IR lighting based on thebrightness of the second image frame being less than or equal to asecond threshold value.

The controlling of the transmittance of the electrochromic device basedon the brightness of the second image frame may include controlling thetransmittance of the electrochromic device to be lower as the brightnessof the second image frame is higher.

The controlling of the transmittance of the electrochromic device basedon the brightness of the second image frame may include controlling thetransmittance of the electrochromic device to be lower as the brightnessdifference is greater.

The instructions, when executed, may further cause the processor tocontrol the electronic device to: turn on the IR lighting in response tothe brightness of the first image frame being greater than or equal tothe second threshold value, based on the first image frame beingacquired with the IR lighting off.

The instructions, when executed, may further cause the processor tocontrol the electronic device to: control the transmittance of theelectrochromic device based on the brightness of the first image frame,based on the first image frame being acquired with the IR lighting off.

The instructions, when executed, may further cause the processor tocontrol the electronic device to: control the transmittance of theelectrochromic device based on the brightness of the second image frame,based on the first image frame is acquired with the IR lighting off.

According to an example embodiment, a method of controlling anelectronic device (e.g., 200) including an electrochromic device mayinclude: acquiring an image using a camera (e.g., 705) of the electronicdevice; based on the image being acquired while an IR lighting (e.g.,710) of the electronic device is on; controlling a transmittance of theelectrochromic device based on a brightness of a first area in the imagenot affected by the IR lighting.

Based on the image being acquired with the IR lighting on, the methodmay include: determining a brightness difference by comparing thebrightness of the first area and a brightness of a second area affectedby the IR lighting; and turning off the IR lighting based on thebrightness difference being less than or equal to a first thresholdvalue.

Based on the image being acquired with the IR lighting off, the methodmay include: determining a brightness of an entire area of the image;turning on the IR lighting based on the brightness of the entire areabeing less than or equal to a second threshold value; and controllingthe transmittance of the electrochromic device based on the brightnessof the entire area.

It should be appreciated that various example embodiments of the presentdisclosure and the terms used therein are not intended to limit thetechnological features set forth herein to particular exampleembodiments and include various changes, equivalents, or replacementsfor a corresponding example embodiment. In connection with thedescription of the drawings, like reference numerals may be used forsimilar or related components. It is to be understood that a singularform of a noun corresponding to an item may include one or more of thethings, unless the relevant context clearly indicates otherwise. As usedherein, “A or B,” “at least one of A and B,” “at least one of A or B,”“A, B or C,” “at least one of A, B and C,” and “A, B, or C,” each ofwhich may include any one of the items listed together in thecorresponding one of the phrases, or all possible combinations thereof.Terms such as “first,” “second,” or “first” or “second” may simply beused to distinguish the component from other components in question, anddo not limit the components in other aspects (e.g., importance ororder). It is to be understood that if an element (e.g., a firstelement) is referred to, with or without the term “operatively” or“communicatively,” as “coupled with,” “coupled to,” “connected with,” or“connected to” another element (e.g., a second element), the element maybe coupled with the other element directly (e.g., wiredly), wirelessly,or via a third element.

As used in connection with various example embodiments of thedisclosure, the term “module” may include a unit implemented inhardware, software, or firmware, or any combination thereof, and mayinterchangeably be used with other terms, for example, “logic,” “logicblock,” “part,” or “circuitry.” A module may be a single integralcomponent, or a minimum unit or part thereof, adapted to perform one ormore functions. For example, according to an example embodiment, themodule may be implemented in the form of an application-specificintegrated circuit (ASIC).

Various example embodiments set forth herein may be implemented assoftware including one or more instructions that are stored in a storagemedium (e.g., the memory (not shown) of FIG. 1 ) that is readable by amachine (e.g., the electronic device 100). For example, a processor(e.g., the processor (not shown) of FIG. 1 ) of the machine (e.g., theelectronic device 100) may invoke at least one of the one or moreinstructions stored in the storage medium, and execute it. This allowsthe machine to be operated to perform at least one function according tothe at least one instruction invoked. The one or more instructions mayinclude a code generated by a complier or a code executable by aninterpreter. The machine-readable storage medium may be provided in theform of a non-transitory storage medium. Here, the “non-transitory”storage medium is a tangible device, and may not include a signal (e.g.,an electromagnetic wave), but this term does not differentiate betweenwhere data is semi-permanently stored in the storage medium and wherethe data is temporarily stored in the storage medium.

According to various example embodiments, a method according to anexample embodiment of the disclosure may be included and provided in acomputer program product. The computer program product may be traded asa product between a seller and a buyer (or purchaser described herein).The computer program product may be distributed in the form of amachine-readable storage medium (e.g., compact disc read only memory(CD-ROM)), or be distributed (e.g., downloaded or uploaded) online viaan application store (e.g., PlayStore™) or between two user devices(e.g., smart phones) directly. If distributed online, at least part ofthe computer program product may be temporarily generated or at leasttemporarily stored in the machine-readable storage medium such as memoryof the manufacturer's server, a server of the application store, or arelay server.

According to various example embodiments, each component (e.g., a moduleor a program) of the above-described components may include a singleentity or multiple entities, and some of the multiple entities may beseparately disposed in different components. According to variousexample embodiments, one or more of the above-described components oroperations may be omitted, or one or more other components or operationsmay be added. Alternatively or additionally, a plurality of components(e.g., modules or programs) may be integrated into a single component.In such a case, according to various example embodiments, the integratedcomponent may still perform one or more functions of each of theplurality of components in the same or similar manner as they areperformed by a corresponding one of the plurality of components beforethe integration. According to various example embodiments, operationsperformed by the module, the program, or another component may becarried out sequentially, in parallel, repeatedly, or heuristically, orone or more of the operations may be executed in a different order oromitted, or one or more other operations may be added.

While the disclosure has been illustrated and described with referenceto various example embodiments, it will be understood that the variousexample embodiments are intended to be illustrative, not limiting. Itwill further be understood by those skilled in the art that variouschanges in form and detail may be made without departing from the truespirit and full scope of the disclosure, including the appended claimsand their equivalents. It will also be understood that any of theembodiment(s) described herein may be used in conjunction with any otherembodiment(s) described herein.

What is claimed is:
 1. An electronic device comprising: a camera; aninfrared (IR) lighting; an electrochromic device; at least oneprocessor; and at least one memory configured to store instructions,wherein, the instructions, when executed, cause the processor to controlthe electronic device to: acquire a first image frame using the camera;turn off the IR lighting, based on the IR lighting being on; acquire asecond image frame using the camera while the IR lighting is off; andbased on the first image frame being acquired with the IR lighting on,control a transmittance of the electrochromic device based on abrightness of the second image frame.
 2. The electronic device of claim1, wherein the instructions, when executed, cause the processor tocontrol the electronic device to: based on the first image frame beingacquired with the IR lighting on, determine a brightness difference bycomparing a brightness of the first image frame and the brightness ofthe second image frame; and in response to the brightness differencebeing greater than or equal to a first threshold value, turn on the IRlighting.
 3. The electronic device of claim 1, wherein the instructions,when executed, cause the processor to control the electronic device: inresponse to the brightness of the second image frame being less than orequal to a second threshold, turn on the IR lighting.
 4. The electronicdevice of claim 1, wherein, for the controlling of the transmittance ofthe electrochromic device based on the brightness of the second imageframe, the instructions, when executed, cause the processor to controlthe electronic device to: control the transmittance of theelectrochromic device to decrease as the brightness of the second imageframe increases.
 5. The electronic device of claim 2, wherein, for thecontrolling of the transmittance of the electrochromic device based onthe brightness of the second image frame, the instructions, whenexecuted, cause the processor to control the electronic device to:control the transmittance of the electrochromic device to decrease asthe brightness difference increases.
 6. The electronic device of claim1, wherein the instructions, when executed, cause the processor tocontrol the electronic device to: based on the first image frame beingacquired with the IR lighting off, turn off the IR lighting in responseto a brightness of the first image frame being greater than or equal toa second threshold value.
 7. The electronic device of claim 6, whereinthe instructions, when executed, cause the processor to control theelectronic device to: based on the first image frame being acquired withthe IR lighting off, control the transmittance of the electrochromicdevice based on the brightness of the first image frame.
 8. Theelectronic device of claim 6, wherein the instructions, when executed,cause the processor to control the electronic device to: based on thefirst image frame being acquired with the IR lighting off, control thetransmittance of the electrochromic device based on the brightness ofthe second image frame.
 9. A method of controlling an electronic devicecomprising an electrochromic device, comprising: acquiring a first imageframe using a camera of the electronic device; based on an infrared (IR)lighting of the electronic device being on, turning off the IR lighting;acquiring a second image frame using the camera while the IR lighting isoff; and based on the first image frame being acquired with the IRlighting on, controlling a transmittance of the electrochromic devicebased on a brightness of the second image frame.
 10. The method of claim9, wherein based on the first image frame being acquired with the IRlighting on, further comprising: determining a brightness difference bycomparing a brightness of the first image frame and the brightness ofthe second image frame; and in response to the brightness differencebeing greater than or equal to a first threshold value, turning on theIR lighting.
 11. The method of claim 9, further comprising: in responseto the brightness of the second image frame being less than or equal toa second threshold, turning on the IR lighting.
 12. The method of claim9, wherein the controlling of the transmittance of the electrochromicdevice based on the brightness of the second image frame comprises:controlling the transmittance of the electrochromic device to decreaseas the brightness of the second image frame increases.
 13. The method ofclaim 10, wherein the controlling of the transmittance of theelectrochromic device based on the brightness of the second image framecomprises: controlling the transmittance of the electrochromic device todecrease as the brightness difference increases.
 14. The method of claim9, wherein based on the first image frame being acquired with the IRlighting off, further comprising: turning on the IR lighting in responseto a brightness of the first image frame being greater than or equal toa second threshold value.
 15. The method of claim 14, wherein based onthe first image frame being acquired with the IR lighting off, furthercomprising: controlling the transmittance of the electrochromic devicebased on the brightness of the first image frame.
 16. The method ofclaim 14, wherein based on the first image frame being acquired with theIR lighting off, further comprising: controlling the transmittance ofthe electrochromic device based on the brightness of the second imageframe.
 17. A method of controlling an electronic device comprising anelectrochromic device, comprising: acquiring an image using a camera ofthe electronic device; and based on the image being acquired while aninfrared (IR) lighting of the electronic device is on, controlling atransmittance of the electrochromic device based on a brightness of afirst area in the image not affected by the IR lighting.
 18. The methodof claim 17, wherein based on the image being acquired with the IRlighting on, further comprising: determining a brightness difference bycomparing the brightness of the first area and a brightness of a secondarea in the image affected by the IR lighting; and in response to thebrightness difference being less than or equal to a first thresholdvalue, turning off the IR lighting.
 19. The method of claim 17, whereinbased on the image being acquired with the IR lighting off, furthercomprising: determining a brightness of an entire area of the image;turning on the IR lighting in response to the brightness of the entirearea being less than or equal to a second threshold value; andcontrolling the transmittance of the electrochromic device based on thebrightness of the entire area.
 20. A non-transitory computer-readablestorage medium storing instructions that, when executed by a processor,cause the electronic device to perform the operations of claim 9.